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Abstract 



o 

(N 

Qh| Motivated by the surging popularity of commercial cloud computing services, a number of recent 

■^ ■ works have studied annotated data streams and variants thereof. In this setting, a computationally weak 

*^^ \ verifier (cloud user), lacking the resources to store and manipulate his massive input locally, accesses a 

powerful but untrusted /jrover (cloud service). The verifier must work within the restrictive data stream- 
ing paradigm. The prover, who can annotate the data stream as it is read, must not just supply the final 
answer but also convince the verifier of its correctness. Ideally, both the amount of annotation from the 
prover and the space used by the verifier should be sub linear in the relevant input size parameters. 

A rich theory of such algorithms — which we call schemes — has started to emerge. Prior work has 
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C/3 I shown how to leverage the prover's power to efficiently solve problems that have no non-trivial stan- 

dard data stream algorithms. However, even though optimal schemes are now known for several basic 
problems, such optimahty holds only for streams whose length is commensurate with the size of the 
data universe. In contrast, many real-world data sets are relatively sparse, including graphs that contain 
only o(«^) edges, and IP traffic streams that contain much fewer than the total number of possible IP 
addresses, 2'^^ in IPv6. 

QQ ' Here we design the first annotation schemes that allow both the annotation and the space usage to 

CO ■ be sublinear in the total number of stream updates rather than the size of the data universe. We solve 

■^ I significant problems, including variations of INDEX, SET-DISJOINTNESS, and FREQUENCY-MOMENTS, 

f^ . plus several natural problems on graphs. On the other hand, we give a new lower bound that, for the 

CO ' first time, rules out smooth tradeoffs between annotation and space usage for a specific problem. Our 

technique brings out new nuances in Merlin- Arthur communication complexity models, and provides a 
separation between online versions of the MA and AMA models. 

c5 ! 1 Introduction 



The surging popularity of commercial cloud computing services has rendered the following scenario in- 
creasingly plausible. A business — call it AliceSystems — processes billions or trillions of transactions a day. 
The volume is sufficiently high that AliceSystems cannot or will not store and process the transactions on 
its own. Instead, it offloads the processing to a commercial cloud computing service. 

The offloading of any computation raises issues of trust. AliceSystems may be concerned about rela- 
tively benign errors: perhaps the cloud dropped some of the transactions, executed a buggy algorithm, or 
experienced an uncorrected hardware fault. Alternatively, AliceSystems may be more cautious and fear that 
the cloud operator is deliberately deceptive or has been externally compromised. Either way, each time 
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AliceSystems poses a query to the cloud, it may demand that the cloud provide not only the answer but also 
some proof that the returned answer is correct. 

Motivated by this scenario, a number of recent works have studied annotated data streams and their 
valiants |l7]|9l[T0l[Il]|2T]|24l. In this setting, a computationally weak verifier (modeling AliceSystems in 
the above scenario), who lacks the resources to store the entire input locally, is given access to a powerful 
but untrusted prover (modeling the cloud computing service). The verifier must execute within the confines 
of the restrictive data streaming paradigm, i.e., it must process the input sequentially in whatever order it 
arrives, using space that is substantially sublinear in the total size of the input. The prover is allowed to 
annotate the data stream as it is read, with the goal of convincing the verifier of the correct answer. The 
streaming restriction for the verifier fits the cloud computing setting well, as the verifier's streaming pass 
over the input can occur while uploading data to the cloud. 

Prior work ||2] 17] H] HO] |22] |24] has provided considerable understanding of the power of annotated 
data streams, revealing a surprisingly rich theory. A number of fundamental problems that possess no non- 
trivial algorithms in the standard streaming model do have efficient schemes when the data stream may 
be annotated by a prover: the term "scheme" refers to an algorithm involving verifier-prover interaction 
as above. By exploiting powerful algebraic techniques originally developed in the literature on interactive 
proofs ifTSl l26l . these works have achieved essentially optimal tradeoffs between annotation size and the 
space usage of the verifier for problems ranging from frequency moments to bipartite perfect matching. 

However, these schemes are only optimal for streams for which the total number of updates is large 
relative to the size of the data universe. In contrast, many real- world data sets are sparse: for example, many 
real-world graphs, though large, contain much fewer than the maximum possible number (2) of edges, and 
IP traffic streams contain much fewer than the total number of possible IP addresses, 2^^^ in IPv6. 

In this paper, we give the first schemes in the annotations model that allow both the annotation size 
and space usage to be sublinear in the number of items with non-zero frequency in the data stream, rather 
than the size of the data universe n. On the negative side, we also give a new lower bound that for the first 
time rules out smooth tradeoffs between annotation size and space usage for a specific problem. The latter 
result is derived from a new lower bound in the Meiiin-Aithur (MA) communication model that may be of 
independent interest. 

1.1 Related Work 

Aaronson and Wigderson ||2] gave a beautiful MA communication protocol for the SET-DISJOINTNESS prob- 
lem (henceforth, DISJ) using algebraic techniques analogous to those in the famous "sum-check protocol" 
from the world of interactive proofs and probabilistically checkable proofs lITSl . Their protocol is nearly op- 
timal, essentially matching a lower bound of Klauck ll22]| . The Aaronson-Wigderson protocol has sei^ved as 
the starting point for many schemes for annotated data streams. We will refer to such schemes as sum-check 
schemes; a typical example is Proposition 14. ll in this work. 

Aaronson ifTI studied the hardness of the INDEX problem in a restricted version of the MA communi- 
cation model, as well as in a quantum variant of this model. His classical model is similar to the online 
MA communication model that we consider. Annotated data streams were introduced by Chaki^abarti et 
al. 171, and studied further by Cormode et al. lH. These two papers gave essentially optimal annotation 
schemes for problems ranging from exact computation of Heavy Hitters and Frequency Moments to graph 
problems like Bipartite Perfect Matching and Shortest s-t Path. Cormode, Thaler and Yi [illi later extended 
the annotations model to allow the prover and verifier to have a conversation, and dubbed this interactive 
model streaming interactive proofs. They demonstrated that streaming interactive proofs can have expo- 
nentially smaller space and communication costs than annotated data streams, and showed that a number of 
powerful protocols from the literature on interactive proofs can be made to work with streaming verifiers; 
in particular, this applies to a powerful general-purpose interactive proof protocol due to Goldwasser, Kalai, 



and Rothblum f20]. Connode, Mitzenmacher, and Thaler lITOl implemented a number of protocols in both 
the annotated data streams and streaming interactive proof settings, demonstrating genuine scalability in 
many cases. In particular, they developed an implementation of the Goldwasser et al. protocol |!20] that 
approaches practicality. Most relevant to our work on annotated data streams, Cormode, Mitzenmacher, and 
Thaler also used sophisticated FFT algorithms to drastically reduce the prover's runtime in the sum-check 
schemes, which we make frequent use of. 

Two recent works have considered variants of the annotated data stream model. Klauck and Prakash f2M 
study a restricted version of the annotations model in which the annotation must essentially end by the final 
stream update. Gur and Raz lIlTI give protocols for a class of problems in a model that is similar to annotated 
data streams, but more powerful in that the verifier has access to both public and private randomness. This 
corresponds to the AMA communication model. We consider protocols in this model in Section l7!2l 

Early work on interactive proof systems studied the power of space-bounded verifiers (the survey by 
Condon ||8l provides a comprehensive overview), but many of the protocols developed in this line of work 
require the verifier to store the input, and therefore do not work in the annotations model, where the verifier 
must be streaming. An exception is work by Lipton [171, who relied on using fingerprinting techniques 
to allow a log-space streaming verifier to ensure that the prover correctly plays back the transcript of an 
algorithm in an appropriate computational model. This approach does not lead to protocols with sublinear 
annotation length. More recently. Das Sarma et al. studied the "best order streaming model," which can be 
thought of as the annotations model where the annotation is restricted to be a permutation of the input 11131 . 

1.2 Overview of Results and Techniques 

We give an informal overview of our results and the techniques we use to obtain them. Throughout, n will 
denote the size of the data universe and m the number of items with non-zero frequency at the end of a data 
stream (we refer to m as the "sparsity" of the stream). A scheme in which the streaming verifier uses at 
most Cy bits of storage and requires at most Ca bits of annotation from the prover is called a (cQ,Cv)-scheme. 
Section |2] defines our models of computation carefully and sets up temiinology. 

Section |3] contains our first set of results. We begin by precisely characterizing the complexity of the 
sparse PointQuery problem — a natural variant of the well-known INDEX problem from communication 
complexity — giving an (A:log?i,3'log?i)-scheme whenever xy > m. We give similar upper bounds for the re- 
lated problems SELECTION and HeavyHitters. We also prove a lower bound showing that any {ca,Cy)- 
scheme for these problems requires CaCy = Q.{m\og{n / m)) , improving by a \og{n/m) factor over lower 
bounds that follow from prior work on "dense" streams. By a dense stream we mean one where n is not 
much larger than m. This \og{n/m) factor may seem minor, but a striking consequence is that the (very) 
sparse INDEX problem — where Alice's ?i-bit string has Hamming weight OiXogn) — has one-way random- 
ized communication complexity that is within a logarithmic factor of its online MA communication com- 
plexity. This implies that no non-trivial tradeoffs between Merlin's and Alice's message sizes are possible 
for this problem; to our knowledge this is the first problem that provably exhibits this phenomenon. 

Our scheme for sparse PointQuery relies on universe reduction: the prover succinctly describes a 
mapping h:[n]^ [r] that maps the input stream, which is defined over the huge data universe [n] , down to a 
derived stream defined over a smaller universe [r\. By design, if the prover is honest and the mapping h does 
not cause "too many collisions," then the answer on the original stream can be determined from the answer 
on the derived stream. We then efficiently apply known schemes for dense streams to the derived stream. 

For our lower bound in Section |3] we give a novel reduction from the standard (dense) INDEX problem 
to sparse INDEX that is tailored to the MA communication model. We then apply known lower bounds for 
dense index. Our technique also gives what is to our knowledge the first polynomial separation between 
the online MA and AMA communication complexities of a specific (and natural) problem. 



For clarity, the remainder of this overview omits factors logarithmic in n and m when stating the costs of 
schemes. Though these factors are important for Section |3](the consequences of our lower bound being most 
significant when n = m®(''), we anticipate that in practice n and m will usually be polynomially related. 

Sections |4]and|5]contain our most interesting and technically involved results, namely, efficient schemes 
for SiZE-m-SET-DlSJOlNTNESS (henceforth, m-DlSJ) and kth Frequency Moments (henceforth, Fu). The 
schemes here are substantially more complex than those in Section |3] and represent the main technical 
contributions of this paper. 

Section lUgives (m^/^,m^/^) -schemes for both problems, but the schemes rely on "prescient" annotation, 
i.e., annotation provided at the start of the stream that depends on the stream itself. The even more com- 
plex schemes of Section |5] eliminate the need for prescient annotation and also achieve much more general 
tradeoffs between annotation length and space usage. Specifically, Section |5] gives {mcv ,Cv)-schemes for 
;n-DlSJ and Fk for any Cy < m. Notice that one recovers the costs achieved in Section |4]by setting c^, = irP''^ . 

These schemes are the first for these problems that allow both the annotation length and space usage to 
be sublinear in m. At a very high level, there are three interlocking ideas that allow us to achieve this. 

1. The first idea is a careful application of universe reduction. We were able to use a simple version of 
this idea to derive the upper bound for the POINXQUERY problem in Section |3] but in the case of DISJ 
and Fk the universe-reduction mapping /j : [«] — )• [r] specified by the prover is more complicated, and 
requires refinement in the form of the additional ideas described below. 

2. The second idea is addressed to ensuring that the prover performed the universe-reduction step in an 
honest manner, in the sense that the answer on the original stream can indeed be detemiined from the 
answer on the derived stream. The difficulty of ensuring P is honest varies depending on the structure 
of the problem at hand. For F^, the verifier has to make sure that the universe-reduction mapping h is 
injective on the items appearing in the data stream. This requires developing an efficient way for V 
to detect collisions under h, even though V does not have the space to store all of the values /j(x,) for 
stream updates Xi. For m-DlSJ, a notion weaker than injectiveness is sufficient. 

3. The third idea pertains to allowing P to specify the universe-reduction mapping h online. That is, 
for many problems it would be much simpler if P could determine the mapping h in advance i.e. if 
P could be prescient, and send h to V at the start of the stream so that V can determine the derived 
"mapped-down" stream on her own (this is the approach taken in Section IDl. When P must specify h 
in an online fashion, additional insight is required. At a high level, our approach is to have P specify 
a "guess" as to the right hash function at the beginning of the steam, and retroactively modify the 
hash function after the stream has been observed. The challenging aspect of this approach is to ensure 
that P's retroactive modification of the hash function is consistent with the observed data stream, even 
though V cannot refer back to the stream to enforce this. 

We exploit similar ideas to allow V to avoid storing the universe-reduction mapping h herself; this 
is the key to achieving general tradeoffs between annotation length and space usage in Section [5] In 
some schemes, storing this mapping h would be the bottleneck in Vs space usage. We show how V 
can store only a partial description of h, and ask P to fill in the remainder of the description when 
necessary. 

Section |6] exploits all of these results, applying them to several graph problems, including counting 
triangles and demonstrating a perfect matching. Our schemes have costs that depend on the number of 
edges in the graph, rather than the total number of possible edges, and demonstrate that the ideas underlying 
our m-DlSJ and Fk schemes are broadly applicable. We state clearly how our schemes improve over prior 
work throughout. 



Section |7] considers a more general stream update model, which allows items to have negative frequen- 
cies. These negative frequencies potentially break the "collision detection" sub-protocol used in the previous 
sections, so we show how to exploit a source of public randomness to allow these protocols to be carried out. 
Essentially, the public randomness specifies a remapping of the input, so that the prover is highly unlikely 
to be able to use negative frequencies to "hide" collisions. Because the protocols of Section |7]require public 
randomness, they work in the AMA communication and streaming models, as opposed to the MA models 
in which all of our other protocols operate. 

2 Models, Notation, and Terminology 

Many of the algorithms (schemes) in this paper use randomization in subtle ways, making it important to 
properly formalize several models of computation. We begin with Merlin-Arthur communication models, a 
topic first studied by Babai, Frankl and Simon |i3|, which we eventually use to derive lower bounds. We then 
turn to annotated data stream models. At the end of the section we set up some notation and terminology for 
the rest of the paper. Some of our discussion in this section borrows from prior work |0. 

2.1 Communication Models 

Let f : X X y — )• {0, 1} be a function, where X and Y are both finite sets. This naturally gives a 2-player 
number-in-hand communication problem, where the first player, Alice, holds an input x^X, and the second 
player. Bob, holds an input j G F. The players wish to compute F{x,y) by executing a (possibly randomized) 
communication protocol that correctly outputs F{x,y) with "high" probability. In Merlin-Arthur communi- 
cation, there is additionally a "super-player," called Merlin, who knows the entire input {x,y), and can help 
Alice and Bob by interacting with them. The precise pattern of interaction matters greatly and gives rise to 
distinct models. Merlin's goal is to get Alice and Bob to output "1" regardless of the actual value oiF{x,y), 
and so Merlin is not to be blindly trusted. 

One important departure we make from prior work is that we allow Merlin to use private random coins 
during the protocol. Most prior work on MA (and AM) communication 131 |22l |23l defined Merlin to be 
deterministic, which does not make a difference in the basic setting. But in this work we are concerned 
with "online MA" models, where the distinction does matter, and these online MA models are in close 
correspondence with the annotated data stream models that are our eventual topic of study. 

MA Communication. In a Merlin-Arthur protocol (henceforth, "MA protocol") for F, Merlin begins by 
sends a help message ^{x,y, rM), using a private random string r^, that is seen by both Alice and Bob. Then 
Alice and Bob (the pair that constitutes the entity "Arthur") run a randomized communication protocol V, 
using a public random string r^, eventually outputting a bit ou(^ {x,y,rA,^)- Importantly, r^ is not known 
to Merlin at the time he sends f). The protocol V is S^-sound and 5c-complete if there exists a function 
f) : X X y X {0, 1}* — > {0, 1}* such that the following conditions hold. 

1. \fF{x,y) = 1 thenPr^„,^^[out^(;c,j,rA,f)(;c,j,rM)) = 0] < 5c. 

2. If /^(^,3;) =OthenV[i' G {0,1}* : Pr,^[out^(^,3;,rA,f)') = 1] < 5,. 

We define err('P) to be the minimum value of max{5s,5f} such that the above conditions hold. Fol- 
lowing 121, we define the help cost hcost(T') to be 1 +raaxx^y,r„ \\){x,y,rM)\ (forcing hcost > 1, even for 
traditional Merlin-free protocols), and the verification cost vcost('P) to be the maximum number of bits 
communicated by Alice and Bob over all x,y and r^- We define y[Kg{F) = min{vcost(P) +hcost('P) : V 
is an MA protocol for F with err(P) < 5}, and MA(f ) = MAi/3(f ). 



Online MA Communication. An online MA protocol is defined to be an MA protocol, as above, but with 
the communication pattern required to obey the following sequence. (1) Input x is revealed to Alice and 
Merlin; (2) Merlin sends Alice a help message \)\{x,rM) using a private random string r^; (3) Input y is 
revealed to Bob; (4) Merlin sends Bob a help message \)2{x,y, vm)', (5) Alice sends a public-coin randomized 
message to Bob, who then gives a 1-bit output. We see this model as the natural MA variant of one-way 
communication, and the analogy with the gradual revelation of a streamed input should be obvious. 

For such a protocol V, we define hcost('P) to be 1 +rm.Xx,yj-„{\\)\{x,rM)\ + \^2{^,y,rM)\) We define 
soundness, completeness, eiT(P), and vcost('P) as for MA. Define MA^(F) = min{hcost('P) + vcost('P) : 
V is an onhne MA protocol for F with err(P) < 5} and write MA^(F) = MA^i-^{F). 

Online AMA Communication. An online AMA protocol is a souped-up version of an online MA proto- 
col, where public random coins can be tossed at the start, before any input is revealed. The number of such 
coin tosses is added to the vcost of the protocol. This models the cost of an initial round of communication 
between Arthur (i.e., Alice -i- Bob) and Merlin. Note that the second public random string, used when Alice 
talks to Bob, does not count towards the vcost. 

On Merlin's Use of Randomness. In an MA protocol. Merlin can deterministically choose a help message 
that maximizes Arthur's acceptance probability. However, Merlin cannot do so in the online MA model, 
because he does not know the entire input when he talks to Alice. This is why we allow Merlin to use 
randomness in these definitions. 

Two recent papers ||7] |24l use "online MA" to mean a more restrictive model where a deterministic 
Merlin talks only to Bob and not to Alice. With Merlin required to be deterministic, this communication 
restriction is irrelevant, as Merlin cannot tell Alice anything she does not already know. However, we permit 
Merlin to be probabilistic, and in this case we do not know that Merlin can avoid talking to Alice. 

As noted earlier, our goal in defining the communication models this way is to closely con^espond to 
annotated data stream models. In many of our online schemes (see, e.g., Section[5]l, the helper provides ini- 
tial annotation that specifies a random "hash" function, h, and the completeness guarantee of the subsequent 
protocol depends crucially on h having "low collision" properties. Since h must be chosen without seeing all 
of the input, such low collision properties cannot be guaranteed by picking a fixed h in advance. However, if 
the helper chooses h at random, then we do have such guarantees for each fixed input, with high probability. 

2.2 Data Stream Models 

We now define our annotated data stream models. Recall that a (traditional) data stream algorithm computes 
a function F of an input sequence x G U'^ , where A'^ is the number of stream updates, and U is some data 
universe, such as {0, 1}'' or [n] = {0, ...,«— 1}: the algorithm uses a limited amount of working memory 
and has access to a random string. The function F may or may not be Boolean. 

An annotated data stream algorithm, or a scheme, is a pair A= (f), V), consisting of a help function 
f) : U^ X {0, 1}* — )• {0, 1}* used by a prover (henceforth, P) and a data stream algorithm run by a verifier, 
V. Prover P provides f)(x, rp) as annotation to be read by V. We think of t) as being decomposed into 
(1)1 , . . . , f)Af), where the function f), : U^ — )• {0, 1}* specifies the annotation supplied to V after the arrival of 
the ith token Xj. That is, f) acts on x (using rp) to create an annotated stream x^''''' defined as follows: 

x^'''' :=(xi,f)i(x,rp),.X2, f)2(x,rp),...,.XA,, f)A,(x,rp)). 

Note that this is a stream over ZY U {0, 1}, of length A'^ + 1^,- 1 f)i(x, rp) | . The streaming verifier V, who uses w 
bits of working memory and has oracle access to a (private) random string ry, then processes this annotated 
stream, eventually giving an output out^(x''''"'',ry). 



Prescient Schemes. The scheme A = (f), V) is said to be 5i-sound and 5c-complete for the function F if 
the following conditions hold: 

1. For all X G U^, we have Pr,^,,^, [out^{x^''''' ,rv) + F(x)] < 5,.. 

2. For all x G U^ , f)' = % , f)'2, • • • , f)^) G ({0, 1}*)^, we have Pr,^, [out^ (x^i^ry) {f (x)} U {±}] < 5,. 

If 5c = 0, the scheme satisfies perfect completeness; otherwise it has imperfect completeness. An output of 
"±" indicates that V rejects P's claims in trying to convince V to output a particular value for f (x). 

We note two important things. First, the definition of a scheme allows the annotation f);(x, rp) to depend 
on the entire stream x, thus modeling prescience: the advice from the prover can depend on data which the 
verifier has not seen yet. Second, P must convince V of the value of F(x) for all x. This is stricter than the 
traditional definitions of interactive proofs and MA communication complexity (including our own, above) 
for decision problems, which place different requirements on the cases F(x) = and f (x) = 1. In Section 
m we briefly consider a relaxed definition of schemes that is in the spirit of the traditional definition. 

We define err(^) to be the minimum value of maxjS^, 5c} such that the above conditions are satisfied. 
We define the annotation length hcost(^) = maxx,r^^,- |f),(x, rp)|, the total size of P's communications, and 
the verification space cost vcost(^) = w, the space used by the verifier V . We say that ^ is a prescient 
(ca,Cv) -scheme if hcost(^) = 0(ca), vcost(^) = 0(cv) and err(^) < ^. 

Online Schemes. We call A= (f), V) a 5-error online scheme for F if, in addition to the conditions in the 
previous definition, each function f), depends only on (;ci , . . . ,;c,). We define error, hcost, and vcost as above 
and say that A is an online {ca,Cv)-scheme if hcost (^) = 0{ca), vcost (^) = 0{cy), and eiT(^) < ^. 

Unlike prior work |]7], we do not always assume that the universe size n and stream length A'^ are poly- 
nomially related; it is possible that logA'^ = o(log«). Therefore we must be much more careful about loga- 
rithmic factors than in prior work. We do assume that N <n always, because our focus is on sparse streams. 

Notice that the help function can be made deterministic in a prescient scheme, but not necessarily so in 
an online scheme. This is directly analogous to the situation for MA and online MA communication models, 
as discussed at the end of Section IZT] 

AMA Schemes. We also consider what we call AMA schemes, where there is a common source of public 
randomness, in addition to the verifier's private random coins. The AMA scheme model is identical to the 
one considered by Gur and Raz 11211 . who referred to it as the "Arthur-Merlin streaming model." 

An online AMA scheme is identical to a (standard) online scheme, except that the data stream algorithm 
and help function both have access to a source of public random bits. The number of random bits used is 
also counted in both the hcost and the vcost of the scheme. 

On Practicality and the Plausibility of Prescience. Although our definition of a scheme allows anno- 
tation to be sent after each stream update, all the schemes we in fact design in this paper only require 
annotation before the start or after the end of the stream. As a practical matter, this avoids the need for 
fine-grained coordination between the annotation and the data stream. 

Online annotation schemes have the appealing property that the prover need not "see into the future" to 
execute them; at any time t, the prover's message only depends on stream updates that arrived before time t. 
While the online restriction appears most natural, prescient schemes may still be suitable in some settings, 
such as when P has akeady seen the full input prior to V beginning to read it. Consider a volunteer computing 
scenario where the verifier farms out many computations to volunteers, and only inspects a particular input 
if a volunteer has already looked at that input and claims to have found something interesting^. In brief, in 
some settings the prover may naturally see the input before the verifier, and in this case a prescient scheme 
will be feasible. 
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2.3 Relationship Between MA Protocols and Schemes 

Any prescient (resp. online) (c„,Cv) -scheme A = {i),V) for a function F can be converted into an MA 
(resp. online MA) protocol for F in the natural way: Merlin sends the output of the /th help function f), to 
Alice — who receives a prefix of the input stream — or Bob, depending on which of the players possesses 
the /th piece of the input. Alice runs the streaming algorithm V on her input as well as any annotation she 
received, and sends the state of the algorithm to Bob. Bob uses this state to continue running V on his input 
and the annotation he received, and then outputs the end result. The hcost of this protocol is at most c„ logN, 
since Merlin has to specify which stream update / each piece of annotation is associated with, and the vcost 
of this protocol is at most c,,. Thus, lower bounds on usual (resp. online) MA communication protocols 
imply related lower bounds on the costs of prescient (resp. onUne) annotated data stream algorithms. 

2.4 Additional Notation and Terminology 

A data stream specifies an input x incrementally. Typically, x can be thought of as a vector (although more 
generally it may represent a graph or a matrix). Each update in the stream is of the form (/, 5) where / G U 
identifies an element of the universe, and 5 G Z describes the change to the frequency of /. The frequency 
of universe item / is defined as fi{x) := L(;,t.4)ex:yt=! ^k- We refer to the vector /(x) = (/i (x), . . . ,/«(x)) as 
the frequency vector of x, where n denotes the size of the data universe. 

We consider several different update models. In the most general update model, the non-strict turnstile 
model, the 5 values may be negative, and so fi may also be negative. In the strict turnstile model, the 5 
values may be negative, but it is assumed that the frequencies fj always remain non-negative. In the insert- 
only model, the 5 values must be non-negative. Orthogonal to these, in the unit-update version of each 
model, the 5 values are assumed to have absolute value 1 . Each of our results applies to a subset of these 
models, and we specify within the statement of each theorem which update models it applies to. 

Throughout, n will denote the size of the data universe, A'^ will denote the total number of stream updates, 
m will denote the total number of items with non-zero frequency at the end of the stream, and M will refer 
to the total number of distinct items that ever appear within some stream update. We will refer to A'^ as the 
length of the stream, to m as sparsity of the stream, and to M as the. footprint of the stream. Notice that it is 
always the case that m<M <N.\n the case of insert-only streams, m = M, but for streams in the (strict or 
general) turnstile models it is possible for m to be much smaller than M. Note also that while we talk about 
"sparse" streams, this refers to the relative size of n and m, not the absolute size. Indeed, we assume that m 
is typically large, too large for V to store the stream explicitly (else the problems can become trivial). 

We often make use of fingerprint functions of streams, which enable a streaming verifier to test whether 
two large streams have the same frequency vector. The verifier chooses a fingerprint function g(x) at random 
from some family of functions satisfying the property that (over the random selection of the function g), 

Pr[g(x)=5(y)|/(x)//(y)]<l^ 

for a parameter p. Typically, g{x) is an element of a finite field of size poly(/7), and hence the number of 
bits required to store the value g(x) (as well as g itself) is 0{[ogp). Further, there are known constructions 
of fingerprint functions where g{x) can be computed in space Oilogp) by a streaming algorithm in the 
non-strict turnstile update model ||7]. 

3 Point Queries, Index, Selection, and Heavy Hitters 

3.1 Upper Bounds 

Our first result is an efficient online annotation scheme for the POINXQUERY problem, a generalization of 
the familiar INDEX problem. 



Definition 3.1. In the PointQuery problem, the data stream x consists of a sequence of updates of the 
form (/, d), followed by an index i, and the goal is to determine the frequency /, (x) = Y.{jtA)ex:jk=i ^k- 

A prescient (logn,logn)-scheme for this problem is trivial as P can just tell V the index i at the start of 
the stream, and V can track the frequency of i while observing the stream. The vcost can be improved to 
O(logm) if V retains a hashed value of i, and tracks the frequency of matching updates. The first scheme 
has perfect completeness, while the second has completeness eiTor polynomially small in m. 

The costs of the scheme below are in terms of the stream sparsity m, and not the stream length A'^ or 
the stream footprint M; this is significant if m <^ M, which is the case, e.g., for the well-known straggler 
and set-reconciliation problems that have been studied in traditional streaming and communication models 
|[T4l[T9l . Our lower bound in Theorem 13. 9 1 shows our scheme is essentially optimal for moderate universe 
sizes, i.e. when the universe size n is sub-exponential in the sparsity m. 

Theorem 3.2. For any pair {ca,Cy) such that Ca -Cy > m, there is an online {caiogn,Cvlogn)-scheme in the 
non-strict turnstile update model for the POINTQuery problem with imperfect completeness. Any online 
{ca,Cv) scheme with c^ > lognfor this problem requires Ca -Cy = D,{mlog{n/m)). 

Proof. V requires P to specify at the start of the stream a hash function h : [n] ^ [c,,] . V requires h to have 
description length 0{ca), rejecting if this is not the case. We define the derived streams x^ G U^ based on h: 
we set xl = Xk iffh{xu) = j, and otherwise. Intuitively, the hash function h partitions the stream updates in 
X into c,, disjoint buckets, and the vector x^ describes the contents of the jth bucket. V maintains fingerprints 
over a field of size poly(n) of each of the Cy different x^ vectors. 

At the end of the stream, given the desired index i , P provides a description of the (claimed) frequency 
vector in the /j(i)th derived stream, /(x'''''). V computes a fingerprint of the claimed frequency vector, and 
compares it to the fingerprint she computed from the data stream, accepting if and only if the fingerprints 
match. Since each x^ is sparse in expectation, the cost of this description can be low: provided h does not 
map more than 0{c^,) items with non-zero frequency to h{i), P can just specify the item id and frequency 
of the items with non-zero frequency in /(x''('^). In this case, the annotation size is just 0{calogn). If P 
exceeds this amount of annotation, V will halt and reject (output _L). 

Soundness follows from the fingerprinting guarantee: if P does not honestly provide x'^^'^ Vs fingerprint 
of x''(') computed from the data stream will not match her fingerprint of the claimed vector of frequencies. 

To show (imperfect) completeness, we study the probability that the output of an honest prover is re- 
jected. This happens only if m(x''('^), the number of non-zero entries in x''''\ is much larger than its ex- 
pectation. By the paii^wise independence of /i, E[m(x''('))] = m{x)/cy = Ca. Thus, by Markov's inequality, 
Pr[m(x''''') > \Oca] < 1/10. So by specifying a hash function chosen at random from a pairwise indepen- 
dent hash family, and then honestly playing back the items that map to the same region as i , P can convince 

V to accept with probability 9/10. 

Notice that V does not need to enforce that P picks the hash function h at random from a pairwise-wise 
independent hash family, as P has no incentive not to pick the hash functions in this way. That is, since 

V will reject if too many items map to the same region as i , it is sufficient for P to pick h at random from 
a pairwise independent hash family in order to convince V to accept with constant probability. But it is 
equally acceptable if P wants to pick h another way; if he does so, P just risks that V will reject with a 
higher probability. 

The lower bound follows from Theorem 13.91 which we prove in Section [l!2l D 



The scheme of Theorem 1X21 vields nearly optimal schemes for the HeavyHitters and SELECTION 
problems, described below. Table [T] summarizes these results and compares to prior work. 



Problem 


Scheme Costs 


Completeness 


Prescience 


Source 


PointQuery 


(log«,log«) 


Perfect 


Prescient 


121 


PointQuery 


(mlogn,log«) 


Perfect 


Online 


m 


PointQuery 


(Q,logn,Cylogn): CaC^ > n 


Perfect 


Online 


m 


PointQuery 


(Q,log«,Cvlogn): CaCy > m 


Imperfect 


Online 


Theoreml3.2l 


Selection 


(c„ log n,c,, log n): c„Cy>n 


Perfect 


Online 


m 


Selection 


{m\ogn,logn) 


Perfect 


Online 


m 


Selection 


(c„log2«,Cvlog«): CaC,, > mlogn 


Imperfect 


Online 


Corollary |3.4| 


^-HeavyHitters 


(0"'logn,^"'log«): c„Cy>n 


Perfect 


Prescient 


m 


(^-HeavyHitters 


(0"*Calog«,Cvlogn): CaCv > n 


Perfect 


Online 


m 


0-HeavyHitters 


(OTlog«,logn) 


Perfect 


Online 


m 


(/)-HeavyHitters 


(0"*c„log«,Cvlog«): CaCv > mlogn 


Imperfect 


Online 


Corollary |3.6| 


(^-HeavyHitters 


(0-'log« + Q,log«,Cvlog«): CaCy > mlogn 


Imperfect 


Online 


Corollary |5.6| 



Table 1: Comparison of our schemes to prior work. For all three problems, ours are the first online schemes 
to achieve both annotation and space usage sublinear in the stream sparsity m when m <^ -y/n, and we strictly 
improve over the online MA communication cost of prior schemes whenever m = o{n). For brevity, we omit 
factors of log^ (m) from the statement of costs of the 0-HeavyHitters scheme due to Corollary [ 



3.1.1 Selection 

Our definition of the SELECTION problem assumes all frequencies fi 
so this definition is only valid for the strict turnstile update model. 



4JkA)-jk 



1 8k are non-negative, and 



Definition 3.3. The Selection problem is defined in terms of the quantity A'^ = Y,ie\n] fi^ the sum of all 
the frequencies. Given a desired rank p (^ [N], output an item j from the stream x = ((71 , 5i), . . . , {jm, Sm)), 
such that L(,,,4):;,<y 5t<p and L{j,Ay.h>j 5k>N-p. 

Corollary 3.4. For any pair {ca,Cv) such that CaCv > mlogn, there is an online (Qlog n,Cylogn)-scheme 
for Selection in the strict turnstile update model. 

The corollary follows from a standard observation to reduce SELECTION to answering prefix sum 
queries, and hence to multiple instances of the PointQuery problem. V treats each stream update (/, 5) 
in the stream x as an update to 0(log?i) dyadic ranges, where a dyadic range is a range of the form 
[72*^,(7 + 1)1^ — 1] for some j and k. Thus, we can view the set of dyadic range updates implied by x 
as a derived stream of sparsity mlogn. Notice we are using the fact that this transformation from the original 
stream of sparsity m results in a derived stream of sparsity at most mlogn; a different derived stream was 
used in Q to address the Selection problem, but the sparsity of that derived stream could be substantially 
larger than the sparsity of the original stream. 

For any /, the quantity 7]- := Y,(j,S):i<i^ can be written as the sum of the counts of 0{logn) dyadic 
ranges. Thus, at the end of the stream P can convince V that item / has the desired T, value by running 
logn PointQuery protocols as in Theorem 13. 2 1 in parallel on the derived stream of sparsity mlogn. The 
verifier's space usage is the same as for a single PointQuery instance on this stream: V fingerprints 
each of the derived streams x^ defined in the proof of Theorem 13.21 and uses these fingerprints in all log n 
instances of the PointQuery scheme. The annotation length is logn times larger than that required for a 
single PointQuery instance because P may have to describe the frequency vectors of up to log n derived 
streams. 

Thus, we get an online (calog^n,c,,log?i)-scheme as long as qc,, = Q.{mlogn). 
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3.1.2 Frequent Items 

Our definition of the (/>-HeavyHitters problem also assumes all frequencies ft := Y,UkAYik=i^^ ^^^ '^^'^" 
negative, and so this definition is only valid for the strict turnstile update model. 

Definition 3.5. The 0-HeavyHitters problem (also known as frequent items) is to list those items / such 
that /,• > ^A'^, i.e. whose frequency of occuixence exceeds a ^ fraction of the total count A'^ = Y.ie\n] ft- 



We give a preliminary result for the (^-HeavyHitters problem in Corollary 13.61 below. We give a 
substantially improved scheme in Section |5] using the ideas underlying our online scheme for frequency 
moments. 

Corollary 3.6. For all Ca,Cy such that CaCy > miogn, there is an online (ca^^^ logn,Cv\ogn)-scheme for 
solving 0-HeavyHitters in the strict turnstile update model. 

Corollary 13. 6l follows from the following analysis. |]7] Theorem 6. 1] describes how to reduce 0-HeavyHitters 
to demonstrating the frequencies of 0(0^') items in a derived stream. Moreover, the derived stream has 
sparsity 0(mlog«) if the original stream has sparsity m. We use the PointQuery scheme of Theorem |3.2| 
As in Corollary 13.41 the annotation length blows up by a factor 0^^ relative to a single PointQuery, but 
the space usage of V can remain the same as in a single PointQuery instance. Hence, we obtain an online 
(Cfl0^' log«,Cvlog«)-scheme for any CaCv > mlogn. 

3.2 Lower Bound 

In this section, we prove a new lower bound on the online MA communication complexity of the {m,n)- 
Sparse INDEX problem. 

Definition 3.7. In the (m,«)-Sparse INDEX problem, Alice is given a vector x G {0, 1}" of Hamming weight 
at most m, and Bob is given an index i. Their goal is to output the value X;. 

We prove our lower bound by reducing the (dense) INDEX problem (i.e. the (m,«)-Sparse INDEX prob- 
lem with m = &{n)) in the MA communication model to the (m,?i)-Sparse INDEX problem for small m. The 
idea is to replace Alice's dense input with a sparser input over a bigger universe, and then take advantage 
of our sparse PointQuery protocol. A lower bound on the online MA communication complexity of the 
dense INDEX problem was proven in [7, Theorem 3.1]; there, it was shown that any online MA communi- 
cation protocol V requires hcost('P) vcost(7^) > n. Combining this with our reduction of the dense INDEX 
problem to the sparse version, we conclude that any protocol for sparse INDEX must be costly. 

Lemma 3.8. ^ Theorem 3.1] Any online MA communication protocol V for the {n,n)-Sparse INDEX 
problem must have hcost('P) vcost('P) = Q,{n). 

Remark 1. The lower bound of Lemma [3^ was originally proved by Chaki^abarti et al. Q in the commu- 
nication model in which Merlin cannot send any message to Alice. However, the proof easily extends to 
our online MA communication model (where Merlin can send a message to Alice, but that message cannot 
depend on Bob's input). 

Theorem 3.9. Any online MA communication protocol V for the {m,n)-Sparse INDEX problem for which 
hcost('P) > \ogn must have hcost(7') vcost(7') = ^(mlog(?i/m)). 

Proof. Assume we have an online MA communication protocol V for (m,?i)-sparse INDEX. We describe 
how to use this online MA protocol for the sparse INDEX problem to design one for the dense INDEX 
problem on vectors of length n' = mlog(«/m). 
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Let k = \og{n/m). Given an input x to the dense INDEX problem, Alice partitions x into n' /k blocks of 
length k, and constructs a 0-1 vector y of Hamming weight n' /k over the universe {0, 1}^" '^^''^ = {0, 1}" as 
follows. She replaces each block B,- with a 1 -sparse vector v,- G {0, 1}^ , where each entry of v,- con^esponds 
to one of the 2^ possible values of block B,. That is, if block B, of a; equals the binary representation of the 
number j G [2*^], then Alice replaces block B,- with the vector ej G {0, 1}^ , where ej denotes the vector with 
a 1 in coordinate j and Os elsewhere. 

Alice now has an n' /k = m-sparse derived input y over the universe {0, 1}". Merlin looks at Bob's input 
to see what is the index i of the dense vector x that Bob is interested in. Merlin then tells Bob the index £ 
such that i = l^{l — \) + j, where B,- is the block that l is located in, and block B, of Alice's input x equals 
the binary representation of the number j G [2*^]. Notice that Merlin can specify I using \ogn bits. If Bob is 
convinced that y^ = \, then Bob can deduce the value of all the bits in block B, of the original dense vector 
X, and in particular, the value of Xj . 

The parties then run the assumed online MA protocol for (wi,n)-Sparse INDEX. The total hcost of 
this protocol is hcost(P) +\ogn = 0(hcost('P)), and the total vcost is vcost('P). Thus, by Lemma [3^ 
hcost('P) vcost('P) = Q.{n') = Q.{m\og{n / m)) as claimed. D 

Theorem 13. 9| should be contrasted with the following well-known upper bound. 

Theorem 3.10. Assume n < m™. Then the one-way randomized communication complexity of the {m,n)- 
Sparse INDEX Problem is 0{m\ogm). 

Proof. Alice chooses a hash function h : [n] — )• [m^] at random from a pairwise independent family and uses h 
to perform "universe reduction". That is, she sends h along with the set S of m values {h{j) :xj = \]. Notice 
h can be specified with 0(log«) = O(mlogm) bits, and S can be specified with 0{m\ogm) bits. Bob outputs 
1 if h{i) G 5, and otherwise. The coiTcctness of the protocol follows from the paii^wise independence 
property of h: if x, = 0, then with high probability l will not collide under h with any j such that xj = I. 
The total cost of this protocol is O(mlogm). D 

3.3 Implications of the Lower Bound 



Our lower bound in Theorem 13. 9 1 has interesting consequences when it is combined with the upper bound 
in Theorem 13. 101 Consider in particular the (m,«)-Sparse INDEX Problem, where n = 2'". Theorem 13. 101 
implies that the one-way randomized communication complexity of this problem is O(mlogm); that is, 
without any need of Merlin, Alice and Bob can solve the problem with O(mlogm) communication. 

Meanwhile, Theorem 13.91 implies that even if Merlin's message to Bob has length Q.{logn) = 0.{m), 
Alice's message to Bob must have length Q.{m\og{n / m) / m) = Q.{m). Indeed, Theorem 13.91 shows that for 
any protocol V, if hcost('P) > log?i = m, then we must have hcost('P) vcost('P) = Q.{m\og{n/m)) = Q.{m^). 
In particular, this means that if hcost (P) = m, vcost (P) must be Q.{m). This trivially implies that for any 
protocol V with hcost('P) less than m, vcost('P) must still be Q.{m); otherwise we could achieve a protocol 
with hcost (P) = m and vcost ("P) = o{m) simply by running V and adding in extraneous bits to the proof to 
bring the proof length up to m. 

Consequently, the online MA communication complexity of this problem is at least Q.{m), which is 
at most a logarithmic factor smaller than the one-way randomized communication complexity. To our 
knowledge, this is the first problem that provably exhibits this behavior. Specifically, this rules out smooth 
tradeoffs between annotation size and space usage in any annotated streaming protocol for the (m,2'")- 
Sparse INDEX Problem. 

Corollary 3.11. The one-way randomized communication complexity of the {m, 2'")-Sparse INDEX Problem 
is 0{mlogm). The online Merlin-Arthur communication complexity is Q.{m). 
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3.3.1 Other Sparse Problems 

A number of lower bounds in |2j are proved via reductions from INDEX that preserve stream length up 
to logarithmic factors. This holds for SELECTION and HeavyHitters, as well as for the problem of 
determining the existence of a triangle in a graph. For all such problems, the lower bound of Theorem 13.91 
implies corresponding new lower bounds for sparse streams, i.e. streams for which m = o{n). We omit the 
details for brevity. 

3.3.2 Separating Online MA and AMA Communication Complexity 



Another implication of Theorem 13. 9 1 is a polynomial separation between online MA communication com- 
plexity and online AMA communication complexity. Indeed, there is an online AMA protocol of cost 
0{y/m) for the (m,2^)-Sparse INDEX Problem, where the O notation hides factors polylogarithmic in m: 
the first message, which consists of public random coins, is used to specify a hash function h : [n] ^ [trr'] 
from a pairwise independent hash family; this message has length 0{logn) = 0{y/rn). With high probabil- 
ity, h is injective on the set {j : Xj = \}. The parties then run the online MA communication protocol of 
Theorem 13. 2l on the inputs /j(x) and h{i) and output the result. The total cost of this protocol is 0{y/m) as 
claimed. In Appendix |Al we in fact show that up to logarithmic factors in m, this online AMA protocol is 
optimal. 

Meanwhile, the lower bound of Theorem 13.91 implies that the online MA communication complexity 
of this problem is 0.(171^1^). Indeed, if we have a protocol V with hcost(P) = m^''^ > logn, Theorem 13.91 
implies that hcost('P) vcost(P) = Q.{mlog{n/m)) = Q.{m^/^), and hence vcost('P) > m^/'^. 

To our knowledge, this is the first such separation between online AMA and online MA communica- 
tion complexity (we remark that polynomial separations between online MA and MAMA communication 
complexity were already known, for problems including INDEX and DISJ ||2l|7l). Indeed, all previous lower 
bound methods that apply to online MA communication complexity, such as the proof of IT] Theorem 3.1] 
and the methods of Klauck and Prakash |[24ll . in fact yield equivalent AMA lower bounds. At a high level, 
the reason is that these methods work via round reduction - they remove the need for Merlin's message. 
They therefore turn any online MA protocol for a function F into an online "A" protocol for F, which is 
really just a one-way randomized protocol without a prover, allowing one to invoke a known lower bound 
on the one-way randomized communication complexity of F. Similarly, they turn an onUne AMA protocol 
for F into an online AA protocol, which is also just a one-way randomized protocol for F. 

The reason Theorem 13.91 is capable of separating online AMA from MA communication complexity 
is that the reduction in the proof of Theorem 13. 9 1 turns an online MA protocol for the (m,«)-Spai-se INDEX 
Problem into an online MA protocol for the (dense) INDEX Problem with related costs. However, the natural 
variant of the reduction applied to an online AMA protocol for the (wi,n)-Sparse INDEX Problem yields an 
online MAMA protocol for the dense INDEX Problem, not an online AMA protocol (see Appendix lAl for 
details). And the dense INDEX Problem has an online MAMA protocol that is polynomially more efficient 
than any online AMA protocol (see e.g. lIllfTTl). 

4 Prescient Schemes for Sparse Disjointness and Frequency Moments 

In this section and the next, we describe schemes for the m-Disjointness (m-DlSJ) and Frequency Moment 
{Fk) problems. These schemes contain the main ideas of the paper. 
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Scheme Costs 


Completeness 


Prescience 


Source 


(mlogm)^/-', (mlogm)^/-'): m = D.{logn) 


Perfect 


Prescient 


Theorem |43J 


{calogn, Cylogn): CaC„ >n 


Perfect 


Online 


171 


{mlogn, logn) 


Perfect 


Online 





{calognlog^m, Cylogn log,,m): q =mc^^^^ 


Imperfect 


Online 


Theorem |5.1| 



Table 2: Comparison of our m-DlSJ schemes to prior work. Ours are the first schemes to achieve annotation 
length and space usage that are both sublinear in m for m <^ a/«, and we strictly improve over the MA 
communication cost (online or prescient) of prior schemes whenever m = o{n). 



4.1 Background: Optimal Schemes for Dense Problems 

We begin with a scheme achieving optimal tradeoffs between annotation length and space usage for a broad 
class of dense problems. Though this scheme follows readily from prior work ||7] |9], we describe it in detail 
for completeness. This scheme is a good example of a sum-check scheme as described in Section [TTl and is 
based on the Aaronson-Wigderson MA protocol for DISJ 121 . 

Proposition 4.1. Let /'^', . . . ,p' denote the frequency vectors of i- data streams, each over the universe 
\n]. Let g be an i-variate polynomial of total degree d over the integers. Let F = ^"^j g{fj ,... ,f- ), and 
let o be an a priori upper bound on \F\. Then for positive integers Ca,Cy with CaC^ > n, there is an online 
{dca{logn + logo) , icv(logn + logo))-scheme for cofnputing F in the non-strict turnstile update model. 

Proof. We work on F^, the finite field with q elements, for a suitably lai^ge prime q; the choice q > 2d{n-\-o)^ 
suffices. V treats each n-dimensional vector /(^' as a Ca x c,, an^ay with entries in F^^, using any canonical 
bijection between [ca] x [c,,] and [n], and interpreting integers as elements of F^, in the natural way. Through 
interpolation, this defines a unique bivariate polynomial f^^'{X,Y) G F^[X,F] of degree q — 1 in X and 
Cv - 1 in F, such that for all x G [ca],y G [c,,], /^■'' {x,y) = f^J'^ {x,y). 

The polynomials /(^' can then be evaluated at locations outside [ca] x [cy], so in the scheme V picks a 
random position r G F^^, and evaluates f^-'\r,y) for all j G [i] and y G [cy]; V can do this using Cy words of 
memory per vector /(^' in a streaming manner ||7] Theorem 4.1]. Let g denote the total-degree-cf polynomial 
over F^ that agrees with g at all inputs in F^. P then presents a polynomial b{X) of degree at most d{ca — 1) 
that is claimed to be identical to l,ye[c,]S{f''^\X^y)^- ■ ■ J''^Kx,y)). 

V checks that b{r) = Y,ye[c\.]S {f^^\fjy) j ■ ■ ■ jf^^Hf^y))- If this sum check passes, then V believes P's 
claim and accepts Lvg[c„1 ^(■^) ^^ the correct answer It is evident that this scheme satisfies perfect complete- 
ness. The proof of soundness follows from the Schwartz-Zippel lemma: if P's claim is false, then 



Pr 



Hr)= I§~(/''M,---,/'H^,3') 



yelcv] 



<d{ca-l)/q. 



D 



4.2 A Prescient Scheme for Sparse Disjointness 

An important special case of the communication problem DISJ is when Alice's and Bob's input sets are 
promised to be small, i.e., have size at most m <^ n. These should be thought of as sparse instances. The 
sparsity parameter m has typically been denoted by the letter k in the communication complexity literature, 
and the problem has typically been refeixed to as ^-DISJ rather than m-DlSJ; we use m rather than k for 
consistency with our notation in the rest of the paper (where m denotes the sparsity of a data stream). 

Among the original motivations for studying this variant is its relation to the clique- vs. -independent-set 
problem introduced by Yannakakis ||27l to study linear programming formulations for combinatorial opti- 
mization problems. More recent motivations include connections to property testing |j4j. A clever protocol 
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of Hastad and Wigderson lfT6l gives an optimal 0{m) communication protocol for m-DlSJ, improving upon 
tiie trivial 0{m\ogn) and the easy O(mlogm) bounds. This protocol requires considerable interaction be- 
tween Alice and Bob, a feature that turns out to be necessary. Recent results of Buhrman et al. [6] and 
Dasgupta et al. |[T2l give tight ©(mlogm) bounds for m-DlSJ in the one-way model. Very recently, Brody et 
al. im and Saglam and Tardos ll25l have given tight rounds- vs. -communication tradeoffs for m-DlSJ. 

Here we obtain the first nontrivial bounds for m-DlSJ in the annotated streams model, and thus also in 
the online MA communication model. 

Definition 4.2. In the m-DlSJ problem, the data stream specifies two multi-sets S, T C [n], with H^Ho, ||r ||o < 
m, where ||5||o denotes the number of distinct items in 5. An update of the form ((0,/),5) is interpreted as 
an insertion of 5 copies of item / into set S, and an update of the form ( ( 1 , /) , 5 ) is interpreted as an insertion 
of 5 copies of item / into T. The goal is to determine whether or not S and T are disjoint. 

Notice Definition 14.21 allows S and T to be multi-sets, but assumes the strict turnstile update model, 
where the frequency of each item is non-negative. 

Theorem 4.3. Assume m > logn. There is a prescient {{mlogmY'^ ,{m\ognif-l^)-scheme for m-DISJ with 
perfect completeness in the strict turnstile update model. In particular, the MA-communication complexity 
ofm-DlS] is 0{{m\ogm)^'^). Any prescient {ca,c^,) protocol requires CaCy = 0.{m). 

Proof. Obviously if S and T are not disjoint, the prescient prover can provide an item / G STl T at the start 
of the stream and the verifier can check that / indeed appears in both S and T. The total space usage and 
annotation length is just 0(log?i) in this case. 

Suppose now that S and T are disjoint. We first recall that a (y^nlogn, v^logn)-scheme for DISJ follows 
from Proposition 14. II with /(') and /(^) set to the indicator vectors of S and T respectively, and g equal to 
the product function. We refer to this as the dense DISJ scheme because its cost does not improve if j^l and 
|r| are both o{n). 

Our prescient scheme for m-Dis J works as follows. At the start of the stream, the prover describes a hash 
function /i : [«] — >• [r], for some smaller universe [r], with the property that h is injective on ^U T. We will 
write h{S) to denote the result of applying h to every member of 5. The parties can now run the dense DISJ 
scheme whereby P convinces V that h{S) and h{T) are disjoint. Given the existence of an injective function 
h, perfect completeness follows from the fact that if S and T are disjoint, so ai^e h{S) and h{T), combined 
with the perfect completeness of the dense DISJ scheme. Soundness follows from the fact that if / G 5n T, 
then h{i) € h{S) n h{T) i.e. if S and T are not disjoint, then the same holds trivially for h{S) and h{T). 

The dense DISJ scheme run on h{S) and h{T) requires annotation length and space usage O(y^logr). 
We now show that, for a suitable choice of r, /"'s description of /i is also limited to O(y^logr) communica- 
tion, balancing out the cost of the rest of the scheme. 

A family of functions F C [r]m js said to be K"-perfect if, for all 5 C [n] with \S\ < K, there exists a 
function h £ T that is injective when restricted to S. Fredman and Komlos ifTSl have shown that for all 
n>r>K, there exists a K"-perfect family T, with 

m<(i+o(i))' ^ 



-\og{l-t{r,K))J' 
where 



t{r,K):= n(l-;) 
For r > 2k", we can use the crude approximation 



-log(l-f(r,fc)) > t{r,K) > (l--)" > e-^""'^' 
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Scheme Costs 


Completeness 


Prescience 


Source 


{k'^Calogn, kcylogn): CaCv > n 


Perfect 


Online 


El 


{mlogn, logn) 


Perfect 


Online 





{k^np-l^logn, knp-l^logn) 


Perfect 


Prescient 


Theorem |4.5| 


(k^mcy ' lognlog^ m, kc ^XognXog ^ m): c,, > 1 


Imperfect 


Online 


Theorem |5.l| 



Table 3: Comparison of our Fjt schemes to prior work. Ours are the first schemes to achieve annotation 
length and space usage that are both sublinear in m for m <SC \fn, and we strictly improve over the MA 
communication cost of prior protocols (online or prescient) whenever m = o(n). 

to obtain the bound \T\ = OiKe^^'I^Xogyi), which imphes 

log 1^1 = OiK'lr), 

for y^ jr = n(log k) and K = D,{logn). 

Let us pick a family T that is (2m)-perfect. Once P and V agree upon such a family T, the prover, 
upon seeing the input sets S and T, can pick h£ T that is injective onSUT. Describing h requires 0{m^/r) 
bits; P sends this to V before the stream is seen, and V stores it while observing the stream in order to run 
the dense DISJ scheme on h{S) and h{T). To balance out this communication with the 0{^/r\ogr) cost of 
running the dense DISJ scheme on h{S) and h{T), we choose r so that 



m 
r 



©(VrlogJ 



This is achieved by setting r = m^l^ /Xog^'^m. The resulting upper bound is that both the annotation length 
and verifier's space usage are O ((mlogni)^'^) . 

The lower bound follows from known lower bounds for dense streams Q. D 

4.3 A Prescient Scheme for Frequency Moments 

We now present prescient schemes for the ^th Frequency Moment problem, F^. 

Definition 4.4. In the Fk problem, the data stream x consists of a sequence of updates of the form (/, 5), and 
the frequency of item / is defined to be ft = '^{jt,5t)ex:jt=i ^^- The goal is to compute F^ = Y,ie[n] fi- 

The idea behind the scheme, as in the case of m-DlSJ, is that P is supposed to specify a "hash function" 
h to reduce the universe size in a way that does not introduce false collisions. However, for F^ it is essential 
that V ensure h is truly injective on the items appearing in the data stream. This is in contrast to m-DlSJ, 
where a weaker notion than injectiveness was sufficient to guai^antee soundness. The fundamental difference 
between the two problems is that for m-DlSJ, collisions only "hurt the prover's claim" that the two sets are 
disjoint, whereas for F^ the prover could try to use collisions to convince the verifier that the answer to the 
query is higher or lower than the true answer 

Theorem 4.5. There is a prescient {k^rn^'^ logn, km ' log n)-scheme for computing F^ over a data stream of 
sparsity m in the strict turnstile update model. This scheme has perfect completeness. Any prescient {ca,Cv) 
protocol requires CaCy = 0.{m). 

Proof. The idea is to have the prover specify for the verifier a perfect hash function h : [n] — )• [r] , where r is to 
be determined later, i.e. P specifies a hash function h such that for all ;c / y appearing in at least one update 
in the data stream, h{x) / h{y). The verifier stores the description of h, and while observing the stream runs 
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the dense F^ scheme of Proposition 14. 1 1 on the derived stream in which each update (/, 5) is replaced with 
the update {h{i),5). 

As discussed above, it is essential that V ensure h is injective on the set of items that have non-zero 
frequency, as otherwise P could try to introduce collisions to try to trick the verifier. To deal with this, we 
introduce a mechanism by which V can "detect" collisions. 

Definition 4.6. Define the problem Injection as follows. We observe a stream of tuples f; = ((;c,-,Z>,), 5,). 
Each ti indicates that 5; copies of item x,- are placed in bucket bi G [r] . We allow 5; to be negative, modeling 
deletions, and refer to the quantity f(j^b) = lli:(xi,bi)=(j,b) ^i ^^ the count of pair {j,b). We assume the strict 
turnstile model, so that for all pairs {j,b) we have f{j.b) > 0. 

We say that the stream is an injection if for every two pairs {j, b) and {j\b) with positive counts, it holds 
that i = /. Define the output as 1 if the stream defines an injection, and otherwise. 

Lemma 4.7. For any CaCv > r, there is an online {ca log r, Cv log r)-scheme for determining whether a stream 
in the strict turnstile model is an injection. 

Proof. Say that bucket b is pure if there is at most one j e [n] such that /(j,i) > 0. The stream defines an 
injection if and only if every bucket b is pure. 

Notice that a bucket b is pure if and only if the variance of the item identifiers mapping to the bucket 
with positive count is zero. Intuitively, our scheme will compute the sum of the these variances across all 
buckets b; this sum will be zero if and only if the stream defines an injection. Details follow. 

Define three r-dimensional vectors m,v,w as follows: 



*, k, I 



ub=Y. f{j- 



b), 



;e["l 



Mn] 

It is easy to see that if bucket b is pure then vl = ut,- wi,. Moreover, if bucket b is impure then v^ < UhWt; 
this holds by the Cauchy-Schwarz inequality applied to the ^-dimensional vectors whose 7th entries are 
y/fij^b) ^nd -J f(j.b) ■ i respectively (the strict inequality holds because for an impure bucket b, the vector 
given by -J f(n,) • j is not a scalar multiple of the vector given by \/f{j.b) )■ Here, we are exploiting the 
assumption that /(y.^) > for all pairs {i,b), as this allows us to conclude that all JYiJh) values are real 
numbers. 

It follows that X!fee[r] ^\ = Lfoe[r] ^b ■ ^b if and only if the stream defined an injection. Both quantities 
can be computed using the "dense" scheme of Proposition 14. 1 1 Notice that each update f, = {{xj,bi),5i) 
contributes independently to each of the vectors u, v, and w, and hence it is possible for V to run the 
scheme of Proposition 14. 1 1 on these vectors as required. This yields an onUne (c„ log r,Cv log r) -scheme for 
the injection problem for any CaCy > r as claimed. D 

Returning to our F^ scheme, P specifies a hash function h claimed to be one-to-one on the set of items that 
appear in one or more updates of the stream x. V verifies that h is injective using the scheme of Lemma 14771 
If this claim is true, then fyt(x) = Fjt(/i(x)), the frequency moment of the mapped-down stream, and P can 
prove this by running the scheme of Q Theorem 4.1] on the derived stream h(x). 

Perfect completeness follows from P's ability to find a perfect hash function just as in Theorem 14.31 
Soundness follows from the soundness of the INJECTION scheme of Lemma l477l in addition to the soundness 
property of the F^ scheme of 171 Theorem 4. 1]. 
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To analyze the costs, note that by using the hash family of Fredman and Komlos 11151 . the annotation 
length and space cost due to specifying and storing the hash function h is 0{m^\ogn/r). The annotation 
length and space cost of the dense F^ scheme of Proposition 14. 1 1 are 0{k^Calogr) and O(^Cvlogr) for any 
CaCv > ''■ The annotation length and space cost of the INJECTION scheme can be set to 0{calogr) and 
O(cvlogr) respectively. Setting r = m^/^ and Ca = c^ = m^^^ yields the desired costs. D 

5 An Online Scheme for Frequency Moments 



We now give an online version of F^ scheme of Theorem 14.51 A simple modification of this scheme yields 
the scheme for m-DlSJ with analogous costs as claimed in Row 4 of Table |2l In addition to avoiding the use 
of prescience, our online scheme avoids requiring V to explicitly store the hash function sent by P, allowing 
us to achieve a much wider range of tradeoffs between annotation size and space usage relative to Theorems 
1431 and 1431 

Theorem 5.1. ForanyCy> 1, there is an online {k^mcv lognlog^ wi, kc^Xognlog^ m)-scheme for Fi^ in 
the strict turnstile model for a stream of sparsity m over a universe of size n. Any online {ca,Cv)-scheme for 
this problem with Ca > logn requires CaC^, = ^{mlog{n/m)). 

Notice that the annotation length is less than mlogn for any c,, = m^^^\ and therefore this protocol is not 
subsumed by the simple "sparse" scheme (second row of Table |3]l in which P just replays the entire stream 
in a sorted order, and V checks this is done correctly using fingerprints. Notice also that the product of the 
space usage and annotation length is k^mcj log^nlog^ m, which is in o{n) for many interesting parameter 
settings. This improves upon the dense sum-check scheme (first row of Table |3]l in such cases. 

5.1 An Overview of the Scheme 

In order to achieve an online scheme, we examine how to construct perfect hash functions such as those used 
in the prescient F^ scheme of Theorem 14.51 Let S be the set of m items with non-zero frequency at the end 
of the stream: we want the hash function to be one-to-one on S. Choose a hash function h at random from 
paii^wise independent hash family mapping [n] to [r], for r to be specified later - this requires just 0(log«) 
bits to specify. We only expect 0{m^ /r) pairs to collide under h, which means that with constant probability 
there will be Oin?' jr) collisions if h is chosen as specified. The final hash function h* is specified by writing 
down h (which takes only O(logn) bits), followed by the items involved in a collision and some special 
locations for them. The total (expected) bit length to specify this hash function is 0(m^log(«)/r). 

In our online F]^ scheme, P will send such an h at the start of the stream. Notice h does not depend on the 
stream itself - it is just a random pairwise independent hash function - so P is not using prescience. P also 
has no incentive not to choose h at random from a pairwise independent hash family, since the only purpose 
of choosing h in this manner is to minimize the number of collisions under h. If P chooses /? in a different 
way, P simply risks that there ai^e too many collisions under h, causing V to reject. 

Now while V observes the stream, she runs the online sum-check scheme for Fyt given in Proposition 14. II 
on the mapped-down universe of size r, using h as the mapping-down function. At the end of the stream, P 
is asked to retroactively specify a hash function h* that is one-to-one on S as follows. P provides a list Lq 
of all items in S that were involved in a collision under h, accompanied by their frequencies. Assuming that 
these items and their frequencies are honestly specified by P, V can compute their contribution to P,t and 
remove them from the stream. By design, h* is then (claimed to be) injective on the remaining items. V can 
confirm this tentatively using the INJECTION scheme of Lemma 14771 

The remainder of the scheme is devoted to making the correctness a certainty by ensuring that the items 
in Lq and their frequencies are as claimed (we stress that while our exposition of the scheme is modular, all 
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parts of the scheme are executed in pai^allel, with no communication ever occurring from V to P). A naive 
approach to checking the frequencies of the items in Lq would be to run |Lo| independent PointQuery 
schemes, one for each item in L; however there are too many items in Lq for this to be cost-effective. 
Instead, we check all of the frequencies as a batch, with a (sub-)scheme whose cost is roughly equal to that 
of a single INJECTION query. 

This (sub-)scheme can be understood as proceeding in stages, with each stage / using a different pairwise 
independent hash function hi to map down the full original input. Say that an item j is isolated by hj if j is 
not involved in a collision under /i, with any other item with non-zero frequency in the original data stream 
X. The goal of stage / is to isolate a large fraction of items which were not isolated by any previous stage. 

A key technical insight is that at each stage /, it is possible for V to "ignore" all items that are not 
isolated at that stage. This enables V to check that the frequencies of all items that are isolated at stage / are 
as claimed. We bound the number of stages that are required to isolate all items if P behaves as prescribed 
- if P reaches an excessive number of stages, then V will simply reject. 

5.2 Details of the Scheme 

Proof of Theorem l5.lt Let r = mcv ■ P sends a hash function h:[n]^ [r] at the start of the stream, claimed 
to be chosen at random from a pairwise independent hash family. While observing the stream, V mns the 
dense online sum-check scheme for F^ given in Proposition 14. 1 I on the mapped-down universe [r\. Let S be 
the set of items with non-zero frequency at the end of the stream. After the stream is observed, P is asked to 
provide a list Lq of all items with nonzero frequency that were involved in a collision, followed by a claimed 
frequency f* for each i (^ Lq. 

Assuming that these items and their frequencies are honestly specified in Lq by P, V can compute 
their contribution Co = LfeLo ft to F^ and then remove them from the stream by processing updates U = 
{(/, —fl) : / € Lq} within the dense F^ scheme, h is injective on the remaining items. V can confirm this 
using the INJECTION scheme of Lemma l4.7l (conditioned on the assumed coixectness of Lq). Thus the dense 
Fjt scheme will output Ci = Lf^Lo fi- Assuming all of Vs checks within the dense Fi^ scheme pass, V outputs 
Co + Ci as the answer. 

The remainder of the scheme is directed towards determining that the frequency of items in Lq are 
correctly reported. We abstract this goal as the following problem. 

Definition 5.2. Define the ^-MultiIndex problem as follows. Consider a data stream xoLq, where o 
denotes concatenation, x is a usual data stream in the strict turnstile model, while Lq is a list of £ pairs 
{i,f*)- Let / be the frequency vector of x. The desired output is 1 if /,• = /)* for all / G Lq, and otherwise. 

We defer our solution to the £-MULTllNDEX problem to Section 1531 For now, we state our main result 
about the problem in the following lemma. 

— 1/2 

Lemma 5.3. For all c,, > 1, £-MULTllNDEX has an online {mcy logn log^ £, Cylognlog^ i)-scheme in 
the strict turnstile update model. 

Analysis of Costs. Let S be the set of items with non-zero frequency when the stream ends. First, we 
argue that if r is the size of the mapped-down universe, and P chooses the hash function h at random from a 
paiiAvise independent hash family, then with probability 9/10, there will be at most lOm^/r items in S that 
collide under g. Indeed, by a union bound, the probability any item / with non-zero count is involved in a 
collision is at most m/r, and hence by linearity of expectation, the expected number of items involved in a 
collision is at most m^ /r. 

So by Markov's inequality, with probability at least 9/10, the total number of items involved in a collision 
will be at most lOm^/r = 0{mcv ) under the setting r = mcj . Conditioned on this event, P can specify 
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the list Lq and the associated frequencies with annotation length 0{mcv logn), and V can use the Multi- 
Index scheme of Lemma [53] with I = 0{mcv ) to verify the frequencies of the items in Lq are as claimed. 
For any c,, > 1, Lemma [531 under this setting of ^ yields an (mc^v'logn-log^ £,Cvlogn-log^^)-scheme. 

Running all of the sum-check schemes (i.e., the INJECTION scheme and the F^. scheme itself) on the 
mapped-down universe requires annotation 0{k^rcl^^\ogr) and space 0{kcv\ogr) for V; in total, this pro- 
vides an online (m^log?i/r + /:^rlogn/cv + kmc:^^^ logn • log^ ni,Cvlogn • log^.M) -scheme. 

Since we set r = mcj , we obtain a online {k^mcv log^ (wi),^Cvlog?ilog^,(m))-scheme for any c,. > 1. 

The lower bound stated in Theorem 15.11 follows from Theorem 13.91 and an easy reduction from the 

(m,«)-sparse INDEX problem. D 

5.3 A Scheme for Multilndex: Proof of Lemma l53l 

Before presenting an efficient online scheme for the ^-MULTllNDEX Problem, we define two "sub"problems, 
which apply a function to only a subset of the desired input. 

Definition 5.4. Define the problem Sub Injection as follows. We observe a stream of tuples ?, = {xi,bj, 5i), 
followed by a vector z G {0, 1}''. As in the INJECTION problem, each f, indicates that 5, copies of item xi are 
placed in bucket b, G [r]. 

We say that the stream defines a subinjection based on z if for every b such that z/, > 1 , for every two 
pairs {x,b) and {y,b) with positive counts, it holds that x = y. The SUBlNJECTlON problem is to decide 
whether the stream defines a subinjection based on z- 

Notice that the Injection problem is a special case of the SubInjection problem with Zi = 1 for all /. 

Lemma 5.5. For any CaCv > r, there is an online {calogr,Cvlogr)-scheme for SUBlNJECTlON in the strict 
turnstile update model. Moreover, for any constant c > 0, this scheme can be instantiated to have soundness 
error \/r^. 



Proof. Define vectors u, Cy, and w exactly as in the proof of Lemma 14771 and observe that the stream defines 
a sub-injection if and only if LfeeWZfeV^ = Y^beW^bUbWb. V can compute both quantities using the dense 
scheme of Proposition 14.11 with the same asymptotic costs as the scheme of Lemma 14.71 The soundness 
error can be made smaller than 1 /r'^ for any constant c by running the scheme of Proposition 14. 1 l over a finite 
field of size poly(r), for a sufficiently fast-growing polynomial in r. D 

We similarly define the problem SUBF2 over a data universe of size n based on a vector z G {0, 1}" 
as T,ieln]Ziff, the sum of squared frequencies of items indicated by z. This too is a low-degree polyno- 
mial function of the input values, and so Proposition 14. 1 1 implies SUBF2 can be computed by an online 
(cq log r,Cv log r) -scheme in the general turnstile update model for any Ca,Cy such that CaC^ > r (and the 
soundness error in this protocol can be made smaller than 1 /r'^ for any desired constant c). 

Online scheme for £-MultiIndex. The scheme can be thought of as proceeding in t stages (t will be 
specified later), although these stages merely serve to partition the annotation: there is no communication 

from V to P during these stages. Each stage j makes use of a corresponding hash function hj : [n] — > [r] 

1/2 
for r = mcv ■ The t hash functions ai^e provided by P at the start of the stream, so that V has access to 

them throughout the stream. Each hj is claimed to be chosen at random from a pairwise independent hash 

family: if they are, then there are unlikely to be too many collisions, so P has no incentive not to choose hj 

at random. Let / denote the vector of frequencies defined by the input stream, and let /("' denote the vector 

satisfying /]■ = f for i £ Lq, and /)• = for i ^ Lq. 

Stage j begins with a list Ly_i of items. We will refer to these items as "exceptions". P provides a new 

list Lj C Ly_i of items which remain exceptions in stage j; P implicitiy claims that no items in Lj_i \Lj 
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collide with some other input items under hash function hj. Let z^-'^ denote the indicator vector of the list of 
buckets corresponding to Ly_i \Lj, i.e. zJ^^!.^ = 1 if / G Ly_i \Lj, and z^-'' entries are otherwise. To check 
that no items in Ly_i \Lj collide under hj, V will use the SuBlNJECTlON scheme based on the indicator 
vector z'^' over the full original input / as mapped by the hash function hj. Note that since the original input 
stream is in the strict turnstile update model, so is the stream on which the SUBlNJECTlON scheme is run (as 
the SubInjection scheme is simply run on the original input stream as mapped by the hash function hj, 
based on the vector z'^'). Note also that Lj_i and Lj are provided explicitly, so V can compute z^^' easilyo 

Having established that the items in Ly_ i \ Lj are no longer exceptions, V also wants to ensure that the 
frequencies of these items were reported correctly in Lq. To do so, V run the SUBf2 scheme over the vector 
f — f* as mapped by hj to r buckets, based on the z.-'^ indicator vector. The result is zero if and only if 
/,. = fp^ for all / where zj^' = 1. 

The stages continue until Lj = 0, and there are no more exceptions. Provided all schemes conclude 
correctly, and the number of stages to reach Lj = is at most t, V can accept the result, and output 1 for the 
answer to the MultiIndex decision problem. 

Lastly, note that V does not need to explicitly store any of the lists Lj. In fact, P can implicitly specify 
all of the lists Lj while playing the list Lq: for each item i ^ Lq, he provides a number j, thereby implicitly 
claiming that / G Ly for / < j, and / Lji for / > j. 

Analysis of costs. If hj is chosen at random from a paiiwise independent hash family, the probability an 

— 1/2 

item / in Lj^i is involved in a collision with the original stream / under hj is 0{m/r) = 0{cv ). Consider 
the probability that any item / survives as an exception to stage t. The probability of this is 0{cv ), and 
summed over all i items, the expected number is 0{£cv ). Invoking Markov's inequality, with constant 
probability it suffices to set t = 0(log^.^) to ensure that we need at most t stages before no more exceptions 
need to be reported. 

— 1/2 

In stage j, the SUBlNJECTlON and SUBf2 schemes cost (wic,, log?i,Cvlog«). Summing over the t 

— 1/2 

Stages, we achieve for any c,, > 1 an (mcv log(«) -log^ (/M),Cvlog(«) -log^ (m))-scheme as claimed in the 
statement of Lemma [ 



Formal Proof of Soundness. The soundness error of the protocol can be bounded by the probability any 
invocation of the SUBlNJECTlON scheme or the SUBf2 scheme returns an incorrect answer. The soundness 
eiTors of both the SUBlNJECTlON scheme and the SuBf2 scheme can be made smaller than ^ for any 
constant c > 0, and therefore a union bound over all t = 0(log^ i) invocations of each protocol implies that 
with high probabihty, no invocation of either scheme returns an incorrect answer. 

5.4 Implications of the Online Scheme for Frequency Moments 

Our online scheme for 7^^ in Theorem 15. II has a number of important consequences. 

Inner Product and Hamming Distance. Chakrabaiti et al. Q point out that computing inner products and 
Hamming Distance can be directly reduced to (exact) computation of the second Frequency Moment F2, and 
so Theorems 14. 5 1 and | 5 . 1 1 immediatelv yield schemes for these problems of identical cost. 

An improved scheme for 0-HeavyHitters. We can use Lemma [531 to yield an onhne scheme for the 
0-HeavyHitters problem. 

Corollary 5.6. For all Ca,Cv such that CaC^ > mlogn, there is an online (cylogn • log^. (m) + ^^^ logn, 
Cylognlog^ {m))-scheme for solving 0-HeavyHitters in the strict turnstile update model. 



For example, V can add one to the corresponding entry of z'-'' for each item that is marked as an exception. This will cause z^-'' 
to count the number of exceptions in each bucket, rather than indicate them, but this does not affect the correctness. 

21 



Corollary l5.6l follows from a similai^ analysis to Corollary 13.61 Q Theorem 6.1] describes how to reduce 
0-HeavyHitters to demonstrating the frequencies of 0(0^') items in a derived stream. Moreover, the 
derived stream has sparsity 0{mlogn) if the original stream has sparsity m. We use the MULTllNDEX 
scheme of Lemma [531 to verify these claimed frequencies. 

Frequency-based functions. Chakiabarti et al. ||7] Theorem 4.5] also explain how to extend the sum-check 
scheme of Proposition 14.11 to efficiently compute arbitrary frequency-based functions, which are functions 
of the form F(x) = Y,ie\n]S{fi{'^)) for ^ri arbitrary g : {—[N]VJ[N]) ^ Z,. A similar but more involved 
extension applies in our setting, by replacing the dense F^ scheme implied by Proposition 14 . 1 1 with the dense 
frequency-based functions scheme of [7. Theorem 4.5]. We spell out the details below, restricting ourselves 
to the prescient case for brevity; an online scheme with essentially identical costs follows by using the ideas 
underlying Theorem 15. II 

Corollary 5.7. Let F{x) = L;g[„] g{fi{x)) be a frequency-based function. Then there is aprescient {N^'^logn, 
N^'^\ogn)-scheme for computing F{x) in the strict unit-update turnstile model. This scheme satisfies perfect 
completeness. 

Proof. We use a natural modification of the frequency -based functions scheme of fTl, Theorem 4.5]. P 
specifies a hash function h at the start of the stream mapping the universe [n] into [A'^^/^]; P chooses h 
to be injective on the set of items that have non-zero frequency at the end of the stream. Using the per- 
fect hash functions of Fredman and Komlos |[T5l . h can be represented with 0{N^ /rlogn) = 0{N^/'^\ogn) 
bits. V stores h explicitly. After the stream is observed, P and V run the 0-HeavyHitters scheme of 
Corollary 15.61 with = N^^l'^. Using the fact that Y.ifi < ^' by setting the parameters of Corollary 15.61 
appropriately we can ensure that this part of the scheme requires annotation length 0{N^'^\ogn) and has 
space cost 0{N^I^\ogn). This scheme also allows V to determine the exact frequencies of the items in H, 
allowing V to compute cont(//) := Y^ieH s{fi{'^)) ^ which gives the contribution of the items in H to the out- 
put F{x). Moreover, whenever V learns the frequency f of an item in i £ H,V treats this as a deletion of f 

occurrences of item /, thereby obtaining a derived stream z in which all frequencies have absolute value at 
mostA^i/4 

P and V now run the polynomial-agreement scheme that was first presented in ||9l Theorem 4.6] on the 
"mapped-down" input h{z) over the universe [A/^^/"*]. For any c^Cy > r, the polynomial agreement scheme can 
achieve cost (Fmax(z)culogn,Cvlog?i), where ^max(z) denotes max, |/,(z)|, the largest frequency in absolute 
value of any item. Setting c,, = N^'"^ and c„ = A'^^''*, we obtain a prescient (A^^/'*log«,A'^^''^log?2)-scheme as 
claimed. V computes the final answer as F{x) = cont(//) +f (/j(z)) — \H\g{0). 

The final issue is that V needs to verify that h is actually injective over the items that appear in x. V can 
accomplish this using the INJECTION scheme of Lemma 14.71 This does not affect the asymptotic costs of 
our scheme, as the INJECTION scheme can support annotation cost Ca log r and space cost Cy log r for any 

CaCy = ^(A^V4). □ 

Finally, we provide one additional corollary, which describes a protocol that will be useful in the next 
section when building graph schemes. 

Theorem 5.8. Let X,Y C [n] be sets with \X\ < \Y\ <m. Then given a stream in the strict turnstile update 

— 1/2 

model with elements ofX and Y arbitrarily interleaved, there is an online {mcv • log(n) -log^ (wi),Cv • 
log(«) -Xog^ [m))-scheme for determining whether X C Y for any c,, > 1. 

Proof. If X %Y ,P can specify an x G X \ F and prove that x is indeed in X and not Y with two point queries 
using the scheme of Theorem 13.21 For the other case, Chakrabarti et al. show how to directly reduce the 
case X C y to computation of frequency moments Q. The claimed costs follow from Theorem 15. II D 

Table IHprovides a comparison of schemes for the Subset problem in the dense and sparse cases. 
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Scheme Costs 


Completeness 


Online/Prescient 


Source 




(|X|log?i,log?i) 


Perfect 


Prescient 


13 




(c„log?i, Cvlog«): CaCv > n 


Perfect 


Online 


m 




{mlogn, logn) 


Perfect 


Online 


m 


1 -l/2i 


{rn)logn, Cylognlog^ m): c^ > 1 


Imperfect 


Online 


Theorem |5. 8 1 



Table 4: Comparison of our Subset scheme to prior work. Ours is the first online scheme to achieve 
annotation length and space usage that are both sublineai^ in m for m <^ y^, and strictly improves over the 
online MA communication cost of prior protocols whenever m = o{n). 



6 Graph Problems 

We now describe some applications of the techniques developed above to graph problems. The main purpose 
of this section is to demonstrate that the techniques developed within the F^ and m-DlSJ schemes are broadly 
applicable to a range of settings. 

We begin with several non-trivial graph schemes that are direct consequences of the Subset scheme of 
Theorem 15. 8 1 Recall that our definition of a scheme for a function F requires a convincing proof of the value 
of F(x) for all values F{x). This is stricter than the traditional definition of interactive proofs for decision 
problems, which just require that if F(x) = 1 then there is some prover that will cause the verifier to accept 
with high probability, and if F{x) = there is no such prover Here, we consider a relaxed definition of 
schemes that is in the spirit of the traditional definition. We require only that a scheme A= {i),V) satisfy: 

1. For all X s.t. F(x) = 1, we have Pr^^,^Jout^(x'i ''"'■, ry) / 1] < 1/3. 

2. For all x s.t. F(x) = 0, f)' = {i)\,i)2, • • • , fljv) ^ ({0, 1}*)^, we have Pr,^, [out^(x''', ry) = 1] < 1/3. 
Theorem 6.1. Under the above relaxed definition of a scheme, each of the problems PERFECT-MATCHING, 

— 1/2 
CONNECTIVITY, a?2cf NON-BIPARTITENESS has an {nlogn + mCy lognlog^. m, Cylognlog^ m)-scheme 

on graphs with n vertices and m edges for all Cy > I. All three schemes work in the strict turnstile update 

model and improve over prior work ifcy = (oilog m) and Cy = o{m). 

Proof. In the case of perfect matching, the prover can prove a perfect matching exists by sending a matching 
A4, which requires nlogn bits of annotation. In order to prove A^ is a valid perfect matching, P needs to 
prove that every node appears in exactly one edge of M, and that TW C £, where E is the set of edges 
appealing in the stream. V can check the first condition by compaiing a fingerprint of the nodes in A^ to a 
fingerprint of the set {!,...,«}. V can check that Al C £" using Theorem 15. 8 1 

In the case of connectivity, the prover demonstrates the graph is connected by specifying a spanning 
tree T . V needs to check T is spanning, which can be done as in ||71 Theorem 7.7], and needs to check that 
T CE, which can be done using Theorem l5.8l 

In the case of non-bipartiteness, P demonstrates an odd cycle C. V needs to check C is a cycle, C has an 
odd number of edges, and that C Q E. The first condition can be checked by requiring P to play the edges 
of C in the natural order The second condition can be checked by counting. The third condition can be 
checked using Theorem 15.81 D 



Counting Triangles. Returning to our strict definition of a scheme, we give an online scheme for counting 
the number of triangles in a graph. 

Theorem 6.2. For any Cy > 1, there is an online (q logn log m, Cvlognlogm)-sche}ne, with imperfect com- 

-1/2 

pleteness, for counting the number of triangles in a graph on n nodes and m edges, where Ca = mncv 
The scheme is valid in the strict turnstile update model. 
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Scheme Costs 


Completeness 


Online/Prescient 


Source 


(c„log?i,c„log?i): CaCy>n^ 
{rp-logn,logn) 
{calog^n,Cylog^n): Ca =mncv 


Perfect 

Perfect 

Imperfect 


Online 
Online 
Online 


m 

Theorem |6.2| 



Table 5: Comparison of prior work to our scheme for counting the number of triangles in a graph 
with n nodes and m edges. For concreteness, notice that by setting c,, = n, Theorem 16.21 achieves a 
(m?i^/^log^?i,?ilog^?i)-scheme, which improves over prior work as long as m ^ rr'/^. 



Proof. Chakrabarti et al. E Theorem 7.4] show how to reduce counting the number of triangles in a graph to 
computing the first three frequency moments of a derived stream. The derived stream has sparsity m{n — 2). 
Using the online scheme of Theorem 15. II to compute the relevant frequency moments of the derived stream 
yields the claimed bounds. D 

The scheme of Theorem l6.2l should be compared to the (?i^,log?i)-scheme from ||7] Theorem 7.2] based 
on matrix multiplication, referenced in Row 2 of Table |5] and the (/j,v) -scheme for any CaCv > n^ from ||7] 
Theorem 7.3], referenced in Row 1 of Table |5] To compare to the former, notice that Theorem 16.2 1 yields 
a (calog^«,Cvlog^«)-scheme with Ca < rp- as long as m < n^JT^,. To compare to the latter, note that in our 
new scheme, CaCv = mncv , which is less than n^ as long as cj < — . In particular, if we set Cy = n, then 
Theorem l6.2l improves over both old schemes as long as m < rv''^. 

Unfortunately, Theorem 16.21 does not yield a non-trivial MA-protocol for showing no triangle exists. 
Indeed, equalizing annotation length and space usage in our new protocol occurs by setting both quantities 
to {mn)^'^. But Q. ((mn)^' ^) < m only when m > n^, which is to say that the MA communication complexity 
of this protocol is always lai^ger than m, a cost that can be achieved by the trivial MA protocol where Merlin 
is ignored and Alice just sends her whole input to Bob. That is, the interest in the new protocol is that it can 
lower the space usage of V to less than m without drastically blowing up the message length of P to n^ as in 
the matrix-multiplication based protocol from Q. 



7 Non-strict Turnstile Update Model 

All schemes in Sections|4]and[5]work in the strict turnstile update model. The reason these schemes require 
this update model is that they use the INJECTION and SuBlNJECTlON schemes of Lemmata 14.71 and 15.51 as 
sub-routines, and these sub-routines assume the strict turnstile update model. 

In this section, we consider two ways to circumvent this issue. To focus the discussion, we concentrate 
on the online F^ protocol of Theorem 15.11 

7.1 An Online Scheme 

One simple method for handling streams in the non-strict turnstile update model is the following. We use the 
scheme of Theorem 15. II but within the SuBlNJECTlON sub-routine, we treat deletions of items in the input 
stream as insertions of items into the derived stream of {xi,bi, 5i) updates. This ensures that the INJECTION 
and SubInjection schemes correctly output 1 if the derived stream is a subinjection (and the remainder of 
the scheme computes the coiTcct answer on the original stream). However it increases the expected number 
of collisions under the universe-reduction mappings hj, from m ■ |L,_ i\/r to M ■ |L,_ 1 1 /r. The result is that 
we achieve the same costs as Theorem 15. II except the costs depend on to the stream footprint M rather than 
the stream sparsity m (see Section l2!4l ). 
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Corollary 7.1. For any Cy > 1, there is a {k^Mcv ■ log («) • log^ (M) , kc^ ■ log {n) ■ log^. (M) ) online scheme 
for Fk in the non-strict turnstile update model over a stream with footprint M over a universe of size n. 

7.2 An Online AMA Scheme 

In this section, we describe an AMA scheme for the INJECTION problem that works in the non-strict turnstile 
stream update model i.e., the input may define a frequency vector where some elements end with negative 
frequency. The scheme for INJECTION of Lemma 1477] breaks down here, since there may be some cases 
where the checks performed by the protocol indicate that a bucket is pure, when this is not the case: can- 
cellations of item weights in the bucket may give the appearance of purity. To address this, we use public 
randomness, thereby yielding an AMA scheme. In essence, the verifier asks the prover to demonstrate the 
purity of each of the r buckets via fingerprints of the bucket contents. However, if we allow the prover to 
choose the fingerprint function, P could pick a function which leads to false conclusions. Instead, V chooses 
the fingerprinting function using public randomness. The players then execute a new INJECTION protocol 
using the data remapped under the fingerprint function, which is intended to convince V of the purity of the 
buckets. This then allows us to construct protocols with costs that depend on the stream sparsity m rather 
than the footprint M as in Corollary 17.11 

In detail, the new AMA scheme proceeds as follows. Consider the INJECTION problem as defined in 
Definition 14.61 but generalized to allow items with arbitrary integer counts. Consider again a bucket b, and 
for 1 < 7 < log« define y^^ to be the frequency vector of the subset of stream updates {xk,b,5k) placing 
items into bucket b, subject to the restriction that the j'th bit of x^ is equal to £. We observe the following 
property: if bucket b is pure, then one of b^^^ and b^^^ must be the zero vector 0, for each j. Moreover, if b 
is not pure, then there exists a j such that both b^^^ and bJ^^ are not the zero vector. 

A natural way to compactly test whether these vectors are equal to zero (probabilistically) is to use 
fingerprinting (discussed in Section I2.4l i. The verifier V could do this unaided for a single bucket, but we 
wish to run this test in parallel for r buckets. At a high level, we achieve this as follows. Given a stream 
of updates {xk,b,5k), we define two vectors z and o of length rlogn, such that each coordinate of z and o 
con^esponds to a (bucket, coordinate) pair {b,j) G [r] x [logn]. In more detail, we will define z and o such 
that for each bucket b and coordinate j G [logn], the {b, j)th entry of z is a fingerprint of the vector b-i^^, and 
the {b,j)th entry of o is a fingerprint of the vector b^^^. 

We choose the fingerprinting functions to satisfy two properties. 

1 . The fingerprint of the all-zeros vector is always 0. This ensures that if all buckets are pure, then the 
inner product of z and o is 0, as Zbj ■ Obj is for all pairs (bj) G [r] x [log?i]. 

2. If there is an impure bucket, then the inner product of z and o will be non-zero with high probability 
over the choice of fingerprint functions. 

Therefore, in order to determine whether the stream defines an injection, it suffices to compute Y,{h.j)e[r]x[\ogn] Ztj • 
Ob.j, which can be computed using Proposition 14. 1 1 with annotation length c^logn and space cost Cylogn for 
any c„ -c,, > rlogn. 

The idea allowing us to achieve the second property is as follows. If bucket b is impure, then there is 
at least one coordinate j G [logn] such that b^^^ and b-'^^ are both not equal to the all-zeros vector 0. By 
basic properties of fingerprints, this ensures that both zt.j and Obj are non-zero with high probability over 
the choice of fingerprint functions. Moreover, we choose the fingerprinting functions in such a way that 
non-zero terms in the sum L(i.;)e[/-]x [logn] ^bj ■ Obj are unUkely to "cancel out" to zero. 

Consequently, we can state an analog of Lemma 14.71 

Lemma 7.2. For any CaCy > rlogn, there is an online {calogn,Cylogn)-scheme for determining whether a 
stream in the non-strict turnstile model is an injection. 
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Proof. Let F^, be a finite field of size q = poly(?i), where the subsequent analysis determines the required 
magnitude of ^. V uses public randomness to choose two field elements a, and j3 uniformly at random from 
¥q. For each bucket b ^ [r], and each coordinate j € [logn], we define two "fingerprinting" functions gh,j,a 
and gbjfi mapping an ?i-dimensional frequency vector F as follows: 



g,j,„(x) = a"(^'°g"+^-)£ 



x/a 



and 

ie[n] 

where each entry x^ of x is treated as an element of F in the natural manner. 

We now (conceptually) construct two vectors z and o of dimension rlog«, where for each {bj) S 
[r] X [log?i], Zbj = gb,j.a{bJ^^) and Obj = gbj{bj~ ). That is, the (^, j)th entry of z equals the fingerprint 
of the frequency vector of items mapping to bucket b with a in the j'th bit of their binary representation. 
Obsei^ve that gbj.ai'^) = gb.j.pi'^) = for all {b,j) G [r] x [logn], as required by Property 1 above. 

We now show that Property 2 holds, i.e. if there is an impure bucket, then the inner product of z and 
o will be non-zero with high probability over the choice of a and j3. In the following, for an item i G [n] 
and bucket Zj G [r], we let fe{b) denote the frequency with which item £ is mapped to bucket b, and we let ij 
denote the j'th bit in the binary representation of £. We can write the inner product of z and o as 

{b,j)e[r]x [log n] 

£ ^nib-log„+j)pn(b-logn+j) ( £ Mb)a') I £ fdb)li'] 

£ ^«{feiog«+;)^«(i.iog«+;) £ fi{b)fe{b)a^^^' 

{b,j)e[r]x[\ogn] {efy.ij=Q/.=[ 

We therefore see that the inner product of z and o is a polynomial in a and j3 of total degree rPrlogn in 
each variable. Moreover, the coefficient of the term ««{/>iog"+;)+''j3«{'>iog"+;)+'" is precisely Mb)-fe{b) if 
ij = and £'■ = 1, and is otherwise. 

Recall that if bucket b is not pure, then there is at least one coordinate j G [log n] , and items £, £' G [n] 
with £j = and £'■ = 1, such that fe{b) ^ and fii(b) ^ 0. The above analysis implies that z • o is a 
non-zero polynomial in a and j8, as the coefficient of o;«(^iog"+i)+'=j8n(''iog«+y)+^ is non-zero. Hence, by the 
Schwartz-Zippel lemma, the probability over a random choice of a and j8 that z • o = is at most n^rlog n/q. 
Setting q to be polynomial in n, there is only negligible probability (over the choice of a and j3) that z • o is 
zero if the stream is not an injection. 

Finally, notice that the verifier can apply the scheme of Proposition 14. II to compute Y.{b,j)e\r\x\\ogn]Zb,j • 
Ob J, as each stream update {x^^b, 8k) can be treated as logn updates to the vectors z and o. For example, if 
the yth bit of ;c^: is 0, then update (jc/:,Zj, 8k) causes zb.j to be incremented by 8k ■ o;"{^iogn+i)+^*. □ 

Applications. We can apply this online scheme to compute Frequency Moments (and Inner Product, Ham- 
ming Distance, Heavy Hitters etc.) over sparse data in the non-strict turnstile update model. The costs 
of the resulting online AMA scheme are similar to the costs of the online schemes for the same problems 
developed in previous sections. The only difference is that we have scaled m up by a log n factor, to ac- 
count for the fact that within the new AMA sub-scheme for INJECTION, we must run the dense protocol of 
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Proposition 14. 1 1 on vectors z and o of length rlogn, rather than on vectors of length r as in prior sections, 
and substitute the bounds from Lemma 17^ For example, the analog of Theorem 15 .H is that for any c,, > 1, 
there is a {k^mcv •log^(?i) -log^. (m),^Cv -log («) -log^ (m)) online AMA scheme for F^. in the non-strict 
turnstile model. 

8 Conclusion 

We have presented a number of protocols in the annotated data streaming model that for the first time allows 
both the annotation length and the space usage of the verifier to be sublinear in the stream spai^sity, rather 
than just the size of the data universe. Our protocols substantially improve on the applicability of prior work 
in natural settings where data streams are defined over very large universes, such as IP packet flows and 
sparse graph data. 

A number of interesting questions remain for future work. The biggest open question is to determine 
the precise dependence on the stream sparsity in problems such as m-DlSJ and frequency moments. When 
setting the annotation length and the space usage of the verifier to be equal, our protocols have cost roughly 
np-/^, where m is the sparsity of the data stream. The best known lower bound is roughly m^/^. We conjec- 
ture that our upper bound is tight up to logarithmic factors, but proving any Merlin-Arthur communication 
lower bound larger than m^l^ will require new lower bound techniques in communication complexity. An- 
other interesting open question is to give improved protocols for multiplying an nxn matrix A by a vector 
X, when A is sparse (i.e., has o{rP-) non-zero entries), but x may be dense. Achieving this would yield im- 
proved protocols for proving disconnectedness, bipartiteness, or the non-existence of a perfect matching in 
a bipartite graph. Cun^ently we do not know of any protocols for these problems that leverage graph sparsity 
in any way. 
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A An Online AMA Lower Bound for (m, 2 v '^). Sparse INDEX 



We prove that the online 0{^/m) protocol for the (m,2v'")-Spai-se INDEX problem is essentially optimal. 
Our lower bound follows from a natural variant of the reduction in Theorem 13.91 That is, we turn an 
online AMA protocol for the (m,2^)-Sparse INDEX Problem into an online MAMA protocol for the dense 
INDEX Problem. We then invoke a lower bound on the online MAMA communication complexity of INDEX 
Problem due to Klauck and Prakash [24] o 

Theorem A.l. The online AMA protocol complexity of the {m, 2^^) -Sparse INDEX problem is Cl{^/m). 

Proof. Let n = 2^. Assume we have an online AMA communication protocol V for (m,«)-sparse INDEX 
with hcost('P) = Q.{y/m). We describe how to use this protocol for the sparse INDEX problem to design one 
for the dense INDEX problem on vectors of length n' = mlog{n/m) = Q. (m^/^). 

Let k = log{n/m). As in the proof of Theorem 13.91 given an input x to the dense INDEX problem, Alice 
partitions x into n' /k blocks of length k, and constructs a vector y of Hamming weight n' /k over a universe 
of size {n' /k) ■ 2*^ as follows. She replaces each block B,- with a 1-sparse vector v,- G {0, 1}^ , where each 
entry of v, corresponds to one of the 2*^ possible values of block B,. That is, if block B, of x equals the binary 
representation of the number j G [2''], then Alice replaces block B, with the vector ey G {0, 1}^ , where ej 
denotes the vector with a 1 in coordinate j and Os elsewhere. 

Thus, Alice now has an n' /k = m-sparse derived input y over a universe of size {n' /k) • 2^ = «. Merlin 
looks at Bob's input to see what is the index l of the dense vector x that Bob is interested in. Merlin then 
tells Bob the index i such that ^ = 2*^(i — 1) + j, where B, is the block that i is located in, and block B,- 
of Alice's input x equals the binary representation of the number j G [2*^]. Notice i can be specified with 
log?! = 0{y/m) bits. 

Alice and Bob's now use the assumed AMA-protocol for sparse disjointness to establish whether y/; = 1. 
If they are convinced of this, then Bob can deduce the value of all the bits in block B, of the original dense 
vector X, and in particular, the value of X; . 

This yields an MAMA protocol for the dense INDEX problem on n' = Q.iirv'l'^) bits. A lower bound 
of Klauck and Prakash |[24l Lemma 7] implies that the online MAMA complexity of this problem is 
Q.({n'yl'^) = Q.{m^l'^). Notice also that the total hcost of our MAMA protocol is 6>(v^ + hcost('P)) = 
0(hcost('P)), while the vcost is 0(vcost('P)). Thus, if hcost('P) = Q.{y/m), it must be the case that vcost 
is Q.{y/m) as well. This trivially implies that for any protocol V with hcost less than y/m, vcost('P) must 
be Q.{^/m). We conclude the online AMA communication complexity of the problem is Q.{m^l^). This 
completes the proof. 

D 



^ Like the lower bound of Lemma [TS] the lower bound of Klauck and Prakash was originally proved in the communication 
model in which Merlin cannot send any message to Alice. However, the proof easily extends to our online MA communication 
model (where Merlin can send a message to Alice, but that message cannot depend on Bob's input). 
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